Multi-modality therapeutic stimulation using virtual objects and gamification

ABSTRACT

A system and method for therapeutic stimulation using virtual objects and gamification, in which multi-modality stimulus is applied using some combination of virtual elements, attention is enhanced by virtue of the user&#39;s active participation, and long-term use is encouraged by virtue of the entertaining nature of the gamification. Depending on configuration, the system and method may comprise a display comprising virtual objects, a light-producing device (other than the display), an audio-producing device such as speakers or headphones, a haptic feedback device such as a vibratory motor, a means for monitoring the user&#39;s attention, and a software application which applies therapeutic stimulation using some combination of the display, the light-producing device, the audio-producing device, and the haptic feedback device.

CROSS-REFERENCE TO RELATED APPLICATIONS

Priority is claimed in the application data sheet to the followingpatents or patent applications, each of which is expressly incorporatedherein by reference in its entirety:

-   -   Ser. No. 18/074,488    -   Ser. No. 17/592,866, now U.S. Pat. No. 11,517,709    -   Ser. No. 17/574,540, now U.S. Pat. No. 11,524,209

BACKGROUND OF THE INVENTION Field of the Art

This disclosure relates to the field of health and wellness therapies,and more particularly to systems and methods for rehabilitative andtherapeutic neurological treatment using computer games and virtualenvironments.

Discussion of the State of the Art

Research increasingly highlights the importance of continuedneurological stimulation throughout all stages of life includingphysical activity, social connection, and frequent cognitive challenge,especially when combined, in preventing early cognitive decline andonset of neurological disorders including dementia. As well, athletesand competitors in various fields such as physical, digital, andcognitive competitions are increasingly seeking well rounded methods ofneurological evaluation and conditioning for tasks directly andindirectly related to their mode of competition.

Recent research on mice suggests that administration of light and soundat frequencies of gamma oscillations (30 Hz to 100 Hz) can help delaythe onset of neurological decline or even cause neurologicalregeneration through gamma entrainment (see, Adaikkan et al., GammaEntrainment Binds Higher-Order Brain Regions and Offers Neuroprotection,2019, Neuron 102, 929-943, Jun. 5, 2019, and Martorell et al.,Multi-sensory Gamma Stimulation Ameliorates Alzheimer's-AssociatedPathology and Improves Cognition, 2019, Cell 177, 256-271, Apr. 4,2019). These studies have suggested that light and/or sound-based gammaentrainment causes physical changes in the brain by stimulatingoscillations in the electrochemical state of neurons in a way thatreduces inflammation and increases synaptic density, and have suggestedthat gamma entrainment using simultaneous application of light and soundhas a greater effect than gamma entrainment using only light or soundindividually. The physical changes observed included reductions inamyloid plaques and tau phosphorylation, and decreases in neuronal andsynaptic losses.

However, the studies performed to date are generalized in nature and donot provide specific systems or methods whereby this knowledge may beapplied to humans. Further, these studies do not suggest any means fortargeting the particular areas of the brain or particular neurologicalfunctionality affected by certain neurological disorders. These studiesalso fail to consider application of gamma entrainment throughstimulation other than light or sound or application of multi-modalgamma entrainment other than simultaneous application of light andsound, or the use of other than physical stimulation transducers such aslight emitting diodes (LEDs). Additionally, these studies do not explorethe effects of treatment regimens of entrainment to frequencies inducingbrain activity other than gamma waves or utilizing a combination offrequencies at various intervals.

Further, as currently known, brainwave entrainment consists of solely ofboring, repetitive listening to certain audio frequencies or staring atflashing physical lights. Thus, brainwave entrainment is a purelypassive, boring activity, which quickly leads to abandonment of theactivity and little or no benefit.

What is needed is a system and method which applies multi-modalitytherapeutic stimulation using virtual objects and/or gamification.

SUMMARY OF THE INVENTION

Accordingly, the inventor has conceived and reduced to practice, Asystem and method for therapeutic stimulation using virtual objects andgamification, in which multi-modality stimulus is applied using somecombination of virtual elements, attention is enhanced by virtue of theuser's active participation, and long-term use is encouraged by virtueof the entertaining nature of the gamification. Depending onconfiguration, the system and method may comprise a display comprisingvirtual objects, a light-producing device (other than the display), anaudio-producing device such as speakers or headphones, a haptic feedbackdevice such as a vibratory motor, a means for monitoring the user'sattention, and a software application which applies therapeuticstimulation using some combination of the display, the light-producingdevice, the audio-producing device, and the haptic feedback device.

In some embodiments, the therapeutic stimulation may be based in part onthe monitoring of the user's attention. In some embodiments, virtualobjects on the display may be used to provide therapeutic stimulation.Some embodiments may comprise additional components such as antherapeutic regimen selector which adjusts the stimulus based on certaininputs, biometric sensors such as an electroencephalograph which may beused to provide inputs to the software application or therapeuticregimen selector. In some embodiments, stimulus may be applied usingcombinations of stimulation from virtual objects on the display andphysical stimulation transducers such as the light-producing device, theaudio-producing device, and the haptic feedback device. In someembodiments, the software application is a virtual reality environment,and the display may be a virtual reality headset or other virtualreality display hardware.

According to a preferred embodiment, a system for therapeuticstimulation using virtual objects, comprising: a computing devicecomprising a memory and a processor; a display; a stimulus managercomprising a first plurality of programming instructions stored in thememory and operating on the processor, wherein the first plurality ofprogramming instructions, when operating on the processor, causes thecomputing device to: receive a regimen for therapeutic stimulation;select one or more stimulation frequencies based on the regimen; receivedata from a virtual reality (VR) application, the data comprising alocation of a virtual object displayed on the display; based on theregimen selected, instruct the VR application to change a visual stateof the virtual object on the display at the selected stimulationfrequencies; and change one or more of the stimulation frequencies basedon feedback, wherein the feedback comprises determination of a user'sattention based on the user's interaction with the virtual object.

According to another preferred embodiment, a method for therapeuticstimulation using virtual objects, comprising the steps of: receiving,at a stimulus manager, a therapeutic stimulation regimen; selecting oneor more stimulation frequencies based on the regimen; receiving datafrom a virtual reality (VR) application, the data comprising a locationof a virtual object displayed on a display; based on the regimenselected, instructing the VR application to change a visual state of thevirtual object on the display at the selected stimulation frequencies;and changing one or more of the stimulation frequencies based onfeedback, wherein the feedback comprises determination of a user'sattention based on the user's interaction with the virtual object.

According to an aspect of an embodiment, the VR application comprising asecond plurality of programming instructions stored in the memory andoperating on the processor, wherein the second plurality of programminginstructions, when operating on the processor, causes the computingdevice to: operate a virtual environment on the computing device, thevirtual environment comprising the virtual object displayed on thedisplay; receive the instruction to change the visual state of thevirtual object; and change the visual state of the virtual object on thedisplay at the selected stimulation frequencies.

According to an aspect of an embodiment, the user's interaction with thevirtual object comprises user eye movement data captured using one ormore eye tracking sensors.

According to an aspect of an embodiment, the regimen comprises one ormore schemes for evaluating one or more attention sub-processes.

According to an aspect of an embodiment, the attention sub-processescomprise at least attention bias, selective attention, dividedattention, sustained attention, and alternating attention.

According to an aspect of an embodiment, the stimulus manager is furtherconfigured to use the user eye movement data to determine an attentionsub-score for each applicable attention sub-process.

According to an aspect of an embodiment, the determination of the user'sattention is based on an aggregation of each of the attentionsub-scores.

According to an aspect of an embodiment, changing one or more of thestimulation frequencies based on the feedback happens in real-time ornear real-time during the user's current regimen.

According to an aspect of an embodiment, the changing of one or more ofthe stimulation frequencies based on feedback is applied to the user'sfuture regimen.

According to an aspect of an embodiment, an external transducer selectedfrom the list of an audio speaker, an audio headphone, a hapticheadband, a vibrating game controller, an ultrasonic transducer, a radiofrequency stimulator, a magnetic stimulator, transcranial direct currentstimulation electrodes, and a deep brain stimulator, wherein at leastone of the selected plurality of stimuli is an operating frequency forthe external transducer.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The accompanying drawings illustrate several embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention according to the embodiments. It will beappreciated by one skilled in the art that the particular embodimentsillustrated in the drawings are merely exemplary, and are not to beconsidered as limiting of the scope of the invention or the claimsherein in any way.

FIG. 1 is a diagram showing an exemplary overall system architecture fora brainwave entrainment system using virtual objects and environments asvisual, stimulation transducers.

FIG. 2 is a diagram showing an exemplary architecture for the brainwaveentrainment manager aspect of the brainwave entrainment using virtualobjects and environments as visual, stimulation transducers.

FIG. 3 is a diagram of an exemplary brainwave entrainment therapy devicethat can be attached to an exercise machine for targeted brainwaveentrainment therapy with attention tracking and virtual objects.

FIG. 4 is a diagram of an exemplary brainwave entrainment therapy systemfor brainwave entrainment therapy with attention tracking and virtualobjects plus external stimulation transducers that allows formulti-modal, multi-intensity treatment.

FIGS. 5A & 5B are a flow diagrams showing an algorithm for selection ofmodalities and routines for brainwave entrainment and application ofbrainwave entrainment using a virtual environment using eye tracking andbiometric feedback to select virtual objects and entrainment routines.

FIG. 6 is a diagram explaining the use of duty cycles and pulse widthmodulations in applying brainwave entrainment.

FIGS. 7-9 (PRIOR ART) explain the application of eye tracking technologyas a means of determining where a user is looking.

FIG. 10 is a diagram showing an embodiment in which on-screen elementsof a display are used to apply brainwave entrainment in conjunction witheye tracking.

FIG. 11 is a diagram showing an exemplary virtual reality environment inwhich virtual objects may be used as visual stimulation transducers.

FIG. 12 is a diagram showing exemplary gamification of brainwaveentrainment in which in-game objects and elements are used as visualstimulation transducers in conjunction with gameplay activities.

FIG. 13 is a block diagram illustrating an exemplary system architecturefor a therapeutic stimulation system using virtual objects andenvironments to provide multi-modal stimulus, according to anembodiment.

FIG. 14 is a block diagram illustrating an exemplary aspect of atherapeutic stimulation system using virtual objects and environments toprovide multi-modal stimulus, an attention evaluator.

FIG. 15 is a flow diagram illustrating an exemplary method forevaluating and scoring an attention level of user, according to anembodiment.

FIG. 16 is a block diagram illustrating an exemplary hardwarearchitecture of a computing device.

FIG. 17 is a block diagram illustrating an exemplary logicalarchitecture for a client device.

FIG. 18 is a block diagram showing an exemplary architecturalarrangement of clients, servers, and external services.

FIG. 19 is another block diagram illustrating an exemplary hardwarearchitecture of a computing device.

DETAILED DESCRIPTION OF THE DRAWING FIGURES

The inventor has conceived, and reduced to practice, A system and methodfor therapeutic stimulation using virtual objects and gamification, inwhich multi-modality stimulus is applied using some combination ofvirtual elements, attention is enhanced by virtue of the user's activeparticipation, and long-term use is encouraged by virtue of theentertaining nature of the gamification. Depending on configuration, thesystem and method may comprise a display comprising virtual objects, alight-producing device (other than the display), an audio-producingdevice such as speakers or headphones, a haptic feedback device such asa vibratory motor, a means for monitoring the user's attention, and asoftware application which applies therapeutic stimulation using somecombination of the display, the light-producing device, theaudio-producing device, and the haptic feedback device.

In some embodiments, the therapeutic stimulation may be based in part onthe monitoring of the user's attention. In some embodiments, virtualobjects on the display may be used to provide therapeutic stimulation.Some embodiments may comprise additional components such as antherapeutic regimen selector which adjusts the stimulus based on certaininputs, biometric sensors such as an electroencephalograph which may beused to provide inputs to the software application or therapeuticregimen selector. In some embodiments, stimulus may be applied usingcombinations of stimulation from virtual objects on the display andphysical stimulation transducers such as the light-producing device, theaudio-producing device, and the haptic feedback device. In someembodiments, the software application is a virtual reality environment,and the display may be a virtual reality headset or other virtualreality display hardware.

Implementations of visual brainwave entrainment to date have beenlimited to passive visual stimulation using physical lights (typicallylight emitting diodes, or LEDs). There is no interactivity or activeengagement with the visual stimulation transducers, which makes theprocess less effective and uninteresting. Further, the visualstimulation transducers, being physical objects, cannot be changed interms of size or shape, cannot be modified in reaction to user feedback,and are limited in terms of colors available, are generally fixed inplace, and additional lights cannot be added to the system withoutphysically connecting (and likely programming) additional lights.

Virtual objects, on the other hand, have none of these limitations, andcan be used as visual stimulation transducers while users are engagedwith an on-screen display. Brainwave entrainment using virtual objectsprovides essentially unlimited variability in terms of stimulator sizes,shapes, colors, movements, rotations, etc., and allows for the use ofmultiple stimulators simultaneously, each with differentcharacteristics. Any change to a virtual object that is perceptible to auser and can be applied at a repeating frequency may be used to applybrainwave entrainment.

Further, gamification changes the brainwave stimulation from passivereceipt of light therapy to active engagement with the visualstimulation objects, wherein the user's brain is actively stimulatedduring the activity, enhancing the effectiveness of the stimulation.Further, as the user is actively engaged with the game, stimulation canbe applied based on where the user's attention is focused.Attention-based stimulation provides opportunities for both directstimulation (e.g., flashing an object at which the user is looking,playing sounds or providing haptic feedback associated with a gameobject or activity that is the object of the user's attention, etc.) andindirect stimulation (e.g., flashing an object in the user's peripheryof vision, playing sounds or providing haptic feedback associated withthe game, but not the object of the user's attention such as abackground element, background music or sounds, etc.). For example, eyetracking technology can be used to determine where the user is lookingon the screen at any given time, and objects at which the user islooking can be used to provide visual stimulation even if the userchanges his or her attention to a different object on the screen. Theuser's attention to objects on the screen can be monitored over time todetermine whether the user is remaining focused on the activity, or isgetting tired and losing focus, and the determined level of userattention can be used to change the type, intensity, directness, andother characteristics of the stimulation. Other means of determining theuser's attention may be used such as assuming that the user's attentionis focused on an object with which the user has just interacted.

Brainwave entrainment using virtual objects may be further enhanced byusing multiple objects, each capable of providing complementary types ofstimulation, and/or by intentionally directing the user's attention toobjects providing certain types of stimulation. For example, if the useris playing a first person shooter (FPS) game that involves shootingattacking aliens, the user's attention will naturally be focused onfinding attacking aliens, aiming at them, and shooting them. As eachalien will be the focus of the user's attention sequentially, the alienat which the user is currently looking may be flashed at appropriatefrequencies and in appropriate colors to provide appropriate brainwavestimulation. Simultaneously, other objects on the screen (or even thebackground) may be selected to provide a complementary visualstimulation in the periphery of the user's vision. Further, brainwaveentrainment using virtual objects may be enhanced by selecting multipletreatment modalities (e.g., light, sound, vibration, electricalstimulation) applied either simultaneously or sequentially, by varyingthe frequency or frequencies of brainwave entrainment (e.g., from about0.5 Hz to about 100 Hz), and by varying the intensity and/or scale ofthe treatment (e.g., from subtle, localized vibrational or electricalstimulation to area-wide, intense stimulation such as high-intensityroom lighting and sound).

Brainwaves are frequencies at which electrical impulses in the brainoccur. Brainwave frequencies change based on the state of consciousnessof the user (e.g., sleeping, awake, dreaming, concentrating, relaxed,contemplative, meditative, irritated, etc.). Generally speaking,brainwaves are divided into five categories with frequencies roughly inthe following ranges.

Delta waves are brainwaves in the general frequency range of 0.1 Hz to 4Hz. Delta waves occur during deep sleep, and indicate a low level ofarousal. Theta waves are brainwaves in the general frequency range of 4Hz to 8 Hz. Theta waves occur in a state between wakefulness and sleep,such as during daydreaming and meditation, and can indicate drowsiness,creativity, or imagination. Alpha waves are brainwaves in the generalfrequency range of 8 Hz to 12 Hz. Alpha waves occur during a wakingstate, but are associated with relaxation, problem solving, analysis,and decision-making. Beta waves are brainwaves in the general frequencyrange of 12 Hz to 30 Hz. Beta waves occur during alertness,concentration, and strenuous mental activities such as solvingmathematical problems and planning for the future. Gamma waves arebrainwaves in the general frequency range of 30 Hz to 44 Hz. Gamma wavesare associated with high-level information processing. There is evidenceof Lambda brainwaves in a range around 47 Hz to 70 Hz, and otherbrainwave entrainment frequencies may be useful up to around 100 Hz.These ranges are approximate, and there is some overlap between them.

There are many promising uses of brainwave entrainment. One promisinguse of brainwave entrainment is to treat and/or prevent epilepsy. Thereis some evidence that epileptic seizures occur when the brain falls intotheta wave activity (approximately 4 Hz to 8 Hz) during normal wakingconsciousness. Normal waking consciousness is typically associated withbeta wave brain activity (12 Hz to 38 Hz). Performing brainwaveentrainment at beta wave frequencies on persons with epilepsy may helpprevent them from falling into theta wave brain activity, thuspreventing seizures.

Another possible use for brainwave entrainment is to reduce agitation byperforming brainwave entrainment at alpha wave frequencies(approximately 8 Hz to 12 Hz). Alpha wave frequencies are those brainwave frequencies between theta wave activity (typically associated withdreaming) and beat wave activity (typically associated withconcentration and learning). Alpha wave frequencies are associated withrelaxation and calmness. Therefore, brainwave entrainment at alpha wavefrequencies may help induce relaxation and calmness.

Many different wave forms and/or pulse widths may be used in deliveringentrainment at the selected frequency or frequencies, regardless of themodality (light, sound, etc.) of the stimulation. Wave forms mayinclude, but are not limited to, rectangular wave forms, sine waveforms, triangular wave forms, and sawtooth wave forms. Pulse widths orduty cycles at any given frequency may be varied across the entire rangeof the frequency period. For example, at a given frequency, the dutycycle of each period of the frequency can be varied from nearly 0%on-time/100% off-time to nearly 100% on-time/0% off-time. Thus, for agiven frequency, the stimulator (e.g., light) can be on and off for anequal amount of time in each period (a 50% duty cycle), mostly on duringeach period (e.g., a 75% duty cycle), or mostly off during each period(e.g., a 25% duty cycle). In these cases, the frequency of thestimulation is the same, but the amount of on-time of the stimulation ineach period of the frequency is different.

Different pulse widths or duty cycles may be useful, depending on thecircumstances. For example, when engaged in a mental task that requiresvisual acuity, a very low or very high duty cycle may be used to flash alight stimulator at a pulse width that can be captured by the human eye,but is not consciously recognizable. The human eye can capture flashesof light as short as 1/200^(th) of a second (equivalent to a frequencyof 200 Hz), possibly shorter, but because of persistence of vision,cannot distinguish between repeated flashes of light at that frequency.Television and computer monitor frame refresh rates are typically 60 Hzor above, as this is a frequency at which persistence of vision makes itdifficult to distinguish between frames. Thus, for example, the flickerof light stimulation at a frequency of 40 Hz and a 50% duty cycle wouldbe easily perceivable by most human beings as each “on” pulse is1/80^(th) of a second long and separated by another “off” time ofanother 1/80^(th) of a second. However, the flicker of light stimulationat the same frequency, but at a 80% duty cycle would likely not beconsciously perceptible, as the “on” time of each period would lastabout 1/50 of a second and the “off” time of each period would lastabout 1/200 of a second. Thus, the “off” time of each period is withinthe limits of capture by the human eye (200 Hz), but would likely not beconsciously perceptible because it is above the average frequencyresolution (60 Hz) of the human eye, and the light would appear to theconscious mind to be on all the time.

In a similar manner, pulse widths or duty cycles may be adjusted to beperceptible to certain cells in the eye but not others. The human eyehas two different types of light receptors: cones and rods. Cones arethe dominant light receptors used under daylight conditions, andreception of light by cones is called photopic vision. Cones are able todistinguish colors, but are less sensitive to lower light intensity andthe persistence of vision of cones is greater (meaning that thefrequency of pulses that can be distinguished by cones is less than forrods). Rods are the dominant light receptors used at night and underlow-light conditions, and reception of light by rods is called scotopicvision. Rods are not able to distinguish colors, but are more sensitiveto lower light intensity and the persistence of vision of rods is less(meaning that the frequency of pulses that can be distinguished by rodsis greater than for cones). Cones are greatly concentrated in the centerof vision (where the person is directly looking) while rods areconsiderably more dominant in the periphery of vision. This differencein the type of light receptors in the eye can be used to advantage whenselecting either a frequency of stimulation or a pulse width/duty cycleof that frequency. Again using the example above where visual acuity isrequired for a mental task, the pulse width or duty cycle of each periodof a brainwave entrainment frequency of light can be selected to beperceptible to rods but not to cones, thus allowing the brainwaveentrainment frequency of light to be perceived by the brain (through therods in the periphery of vision which have a greater frequencyresolution), but not consciously perceptible to the person (who isprimarily focused on the light received by the cones (in the center ofvision and with a lesser frequency resolution). One or more differentinventions may be described in the present application. Further, for oneor more of the inventions described herein, numerous alternativeembodiments may be described; it should be appreciated that these arepresented for illustrative purposes only and are not limiting of theinventions contained herein or the claims presented herein in any way.One or more of the inventions may be widely applicable to numerousembodiments, as may be readily apparent from the disclosure. In general,embodiments are described in sufficient detail to enable those skilledin the art to practice one or more of the inventions, and it should beappreciated that other embodiments may be utilized and that structural,logical, software, electrical and other changes may be made withoutdeparting from the scope of the particular inventions. Accordingly, oneskilled in the art will recognize that one or more of the inventions maybe practiced with various modifications and alterations. Particularfeatures of one or more of the inventions described herein may bedescribed with reference to one or more particular embodiments orfigures that form a part of the present disclosure, and in which areshown, by way of illustration, specific embodiments of one or more ofthe inventions. It should be appreciated, however, that such featuresare not limited to usage in the one or more particular embodiments orfigures with reference to which they are described. The presentdisclosure is neither a literal description of all embodiments of one ormore of the inventions nor a listing of features of one or more of theinventions that must be present in all embodiments.

Headings of sections provided in this patent application and the titleof this patent application are for convenience only, and are not to betaken as limiting the disclosure in any way.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or morecommunication means or intermediaries, logical or physical.

A description of an embodiment with several components in communicationwith each other does not imply that all such components are required. Tothe contrary, a variety of optional components may be described toillustrate a wide variety of possible embodiments of one or more of theinventions and in order to more fully illustrate one or more aspects ofthe inventions. Similarly, although process steps, method steps,algorithms or the like may be described in a sequential order, suchprocesses, methods and algorithms may generally be configured to work inalternate orders, unless specifically stated to the contrary. In otherwords, any sequence or order of steps that may be described in thispatent application does not, in and of itself, indicate a requirementthat the steps be performed in that order. The steps of describedprocesses may be performed in any order practical. Further, some stepsmay be performed simultaneously despite being described or implied asoccurring non-simultaneously (e.g., because one step is described afterthe other step). Moreover, the illustration of a process by itsdepiction in a drawing does not imply that the illustrated process isexclusive of other variations and modifications thereto, does not implythat the illustrated process or any of its steps are necessary to one ormore of the invention(s), and does not imply that the illustratedprocess is preferred. Also, steps are generally described once perembodiment, but this does not mean they must occur once, or that theymay only occur once each time a process, method, or algorithm is carriedout or executed. Some steps may be omitted in some embodiments or someoccurrences, or some steps may be executed more than once in a givenembodiment or occurrence.

When a single device or article is described herein, it will be readilyapparent that more than one device or article may be used in place of asingle device or article. Similarly, where more than one device orarticle is described herein, it will be readily apparent that a singledevice or article may be used in place of the more than one device orarticle.

The functionality or the features of a device may be alternativelyembodied by one or more other devices that are not explicitly describedas having such functionality or features. Thus, other embodiments of oneor more of the inventions need not include the device itself.

Techniques and mechanisms described or referenced herein will sometimesbe described in singular form for clarity. However, it should beappreciated that particular embodiments may include multiple iterationsof a technique or multiple instantiations of a mechanism unless notedotherwise. Process descriptions or blocks in figures should beunderstood as representing modules, segments, or portions of code whichinclude one or more executable instructions for implementing specificlogical functions or steps in the process. Alternate implementations areincluded within the scope of embodiments of the present invention inwhich, for example, functions may be executed out of order from thatshown or discussed, including substantially concurrently or in reverseorder, depending on the functionality involved, as would be understoodby those having ordinary skill in the art.

Definitions

The term “amplitude” means the difference between the high or low stateof a signal or wave form and the base state of that signal or wave formin a full period (high/low or on/off cycle) of the frequency of thesignal or wave form.

The term “biometrics” as used herein mean data that can be input,directly measured, or computed using directly measured data from a user.This data includes but is not limited to physical and virtual movement,physiological, biological, behavioral, navigational, cognitive,alertness and attention, emotional, and brainwave measurements andpatterns.

The phrase “brainwave entrainment” means application of a stimulus witha frequency from about 0.5 Hz to about 100 Hz as a means of neurologicaltherapy. The stimulus may be of any perceptible form such as, but notlimited to, light, sound, vibration, or electrical stimulation. Thestimulus need not be from the same source (e.g., two light sources eachat 20 Hz could be synchronized to produce a 40 Hz stimulus) or from thesame modality (e.g., a sound source at 15 Hz and a light source at 15 Hzcould be synchronized to produce a 30 Hz stimulus).

The term “conditioning” as used herein means all aspects of the systemthat can be used for the improvement, training, treatment of or exposureto aspects of neurological functioning. This could be in the form of aprescribed regimen from an expert, recommendation algorithm,self-selected experiences, or combination thereof.

The term “display” means any type of device capable of producing anoutput visible to a user of the system. A non-limiting list of displaysincludes televisions, computer monitors, tablet and mobile phonescreens, VR headsets, and projectors.

The phrase “duty cycle” means the amount of time that a frequency signalis in the “high” or “on” state, expressed as a percentage, wherein eachfull period (complete high/low cycle) of the frequency signal represents100%. Note that “duty cycle” and “pulse width” are two different meansof expressing the same concept.

The term “expert” as used herein means an individual with specializationin an area via formal training, credentials, or advanced proficiency ina modality of interest to the user or with regard to neurologicalfunctioning. This includes but is not limited to physicians,psychiatrists, physical therapists, coaches, fitness trainers, highlevel athletes or competitors, and teachers.

The term “frequency” means a signal or wave form having a periodicrepetition of high/low or on/off states. Examples of signals and waveforms that exhibit the characteristic of frequency include, but are notlimited to, rectangular wave forms, sine wave forms, triangular waveforms, and sawtooth wave forms.

The terms “game” or “game application” mean any computer game, puzzle,display, animation, or simulation comprising virtual objects that can beinteracted with in some manner by a person. These phrases include, butare not limited to, traditional two-dimensional games and puzzles,three-dimensional virtual reality (VR) applications and environments,enhanced reality and augmented reality applications and environments(comprising both real-world elements and virtual elements, such asvirtual objects superimposed on a video feed of the real environmentsurrounding the user), and interactive applications that allow one tosense virtual objects through haptic feedback (whether or not associatedwith a visual display of the objects).

The term “gamification” as used herein means the application ofbrainwave entrainment using a game or a game application.

The phrases “neurological functioning” and “neurological function” asused herein mean any and all aspects of neuroscience and neurology whereinput, output, processing, or combination thereof involve aspects of thenervous system. These include but are not limited to functional as wellas anatomical aspects of cognitive, sensory, motor, emotional, andbehavioral functions and experiences.

The phrase “pulse width” means the amount of time that a frequencysignal is in the “high” or “on” state, expressed as a time period thatis a portion of each full period (complete high/low cycle) of thefrequency signal. Note that “duty cycle” and “pulse width” are twodifferent means of expressing the same concept. The phrase “pulse widthmodulation” is often used to denote changing of the pulse width of afrequency signal.

The term “transducer” as used herein means a device that converts anelectrical signal into variations in a physical quantity, such as sound,light, pressure, or electrical stimulation. A display is included in thedefinition of “transducer.”

The phrase “stimulation transducer” as used herein means a transducerused to stimulate one of the senses of a person or animal. Any portionof a display may be used as a stimulation transducer, non-limitingexamples of which include virtual objects or backgrounds on the display.

The phrase “virtual object” means a computer-generated simulation of anobject perceivable to a human being. Virtual objects include, but arenot limited to, visible virtual objects such as two-dimensional andthree-dimensional shapes shown on a display, non-visible virtual objectssuch as those that might be “felt” through haptic feedback (e.g., glovesequipped with haptic feedback equipment that provide resistance to theuser's fingers around the contours of a virtual object in space), andany combination of the two (e.g., a visible virtual object displayed ina virtual reality environment through a VR headset which can also be“felt” by the user via haptic feedback). A virtual object does not haveto be gamified and may be, for example, a virtual object displayed on ascreen.

Conceptual Architecture

FIG. 1 is a diagram showing an exemplary overall system architecture 100for a brainwave entrainment system using virtual objects andenvironments as visual stimulation transducers. In this embodiment, thesystem comprises a brainwave entrainment manager 200, a virtual reality(VR) application 140, a therapy regimen controller 110, one or morespatial sensors 130, one or more biometric sensors 120, and one or moreexternal transducers, and a display 160.

The brainwave entrainment manager 200 is the core of the system, andmanages inputs from, and outputs to, other components of the system. Itis responsible for selection of entrainment routines, evaluation of theuser's attention, and activation of both virtual and physicalstimulation transducers.

The therapy regimen controller 110 is an administrative interface thatallows an administrator (e.g., a physician, therapist, masseuse, orother service provider) to select therapy regimens for application tothe user (who may be a patient, client, etc., of the administrator). Thetherapy regimen controller 110 may be used, for example, to select aregimen for brainwave entrainment that emphasizes alpha wave stimulationto induce relaxation in an overstimulated user.

The biometric sensors 120 are sensors that measure a physical orphysiological characteristic of the user, such as heart rate,temperature, sweat production, brain activity (using anelectroencephalograph, or EEG), etc. Biometric sensors 120 are used toprovide feedback to the brainwave entrainment manager 200 as to thephysical or physiological state of the user, which may be used to inferthe user's mental state. For example, a biometric sensor 120 thatmeasures the user's heart rate may be used to infer the user's level ofrelaxation (or lack thereof), thus providing feedback as to theeffectiveness of alpha brainwave entrainment intended to inducerelaxation.

Spatial sensors 130 are sensors that measure a user's physical locationin space or a location at which the user is focusing his or herattention. For two dimensional screens, eye movement may be tracked andthe location of the user's gaze may be calculated. In the case ofvirtual reality (VR), the user's body may be tracked, or if the user iswearing a VR headset, the orientation of the headset can be used todetect the user's head movements. Spatial sensors 130 are used to detectthe user's engagement with virtual objects and virtual environments,such that brainwave entrainment using those objects and environments canbe adjusted, accordingly.

The VR application 140 is used for gamification of brainwaveentrainment. While a VR application 140 is shown here, in principle anycomputer game, puzzle, display, or animation can be used, whetherinteractive or not, and whether three-dimensional or two-dimensional.The VR application 140 can be a specially-designed program intended foruse with the system, or can be an off-the-shelf game or applicationadapted for use with the system. In either case, the VR application 140will either have an interface with the brainwave entrainment manager200, or will have a brainwave entrainment manager 200 integrated intoit, whereby the brainwave entrainment manager 200 is used to controlbrainwave entrainment using the virtual objects in the VR application140.

The external transducers 150 are physical stimulation transducers thatmay be used to complement brainwave entrainment using virtual objects. Anon-limiting list of external transducers 150 includes lights or LEDs,speakers or other audio-producing devices, vibratory or otherpressure-producing devices, and electrical stimulators. As an example,while brainwave entrainment is being applied visually using virtualobjects on a screen, the brainwave entrainment may be supplemented orcomplemented by audible brainwave entrainment using speakers.

The display 160 may be any type of display producing an output visibleto a user of the system. A non-limiting list of displays 160 includescomputer and tablet screens, VR headsets, and projectors. The display160 is the means by which visual brainwave entrainment may be appliedusing virtual objects.

FIG. 2 is a diagram showing an exemplary architecture for the brainwaveentrainment manager aspect of the brainwave entrainment using virtualobjects and environments as visual stimulation transducers. In thisembodiment, the brainwave entrainment manager 200 comprises anentrainment routine selector 201, an attention evaluator 202, an in-gameobject activator 203, and an external transducer activator 204. Theentrainment routine selector 201 receives input VR application input,therapy regimen controller input, and biometric sensor input, and inputfrom the attention evaluator 202. Based on those inputs, the entrainmentroutine selector chooses and/or modifies a brainwave routine appropriatefor the circumstances. For example, if the therapy regimen controllerinput specifies that the overall brainwave entrainment goal isrelaxation, the entrainment routine selector 201 may select alpha waveentrainment as the primary entrainment therapy, and may choose to applyalpha wave entrainment to a background virtual object, as flashing ofbackground objects will be less intrusive (and possibly more relaxing)to the user than flashing of objects to which the user's attention isdirected. To determine which objects are not the subject of the user'sattention, the attention evaluator 202 receives input from a spatialsensor (e.g., a camera used to track eye movements) to determine wherethe user is looking on the screen at a given moment. The entrainmentroutine selector 201 then modifies the entrainment routine to flash anobject or objects at which the user is not looking using an in-gameobject activator 203 which interfaces with the VR application toidentify which objects should be flashed.

The user's attention need not be tracked via a camera, and may betracked through other means. For example, the user's attention may betracked by monitoring the user's interaction with the virtual objects orvirtual environment in the form of mouse clicks, keyboard activity,orientation of the user's head or body (e.g., when a virtual realityheadset is being used), orientation and/or movement of hand-heldtrackable devices such as game controllers with integratedaccelerometers, gyroscopes, etc. In some embodiments, the user'sattention may be tracked not in terms of visual direction or attention,but in the more general sense of focus, consistency, ability toconcentrate, level of interest, response times, or other factors notnecessarily associated with the direction of the user's vision. All ofthese things may be incorporated into decisions by the entrainmentroutine selector 201 as to changes to be made to the entrainmentroutine.

Simultaneously, the entrainment routine selector 201 may activate one ormore external transducers 204 using an external transducer activator204, where the entrainment routine selector 201 determines that externaltransducers may supplement or complement the brainwave entrainment usingvirtual objects. The entrainment routine selector 201 may further usefeedback to determine whether the selected entrainment routine is havingthe desired effect. As an example, the entrainment routine selector 201may use biometric feedback such as a user's heart rate (e.g., a loweringheart rate may be used to infer relaxation) to change the entrainmentroutine. For example, a lowering heart rate during alpha waveentrainment would likely indicate relaxation, in which case theentrainment routine would remain unmodified, but a rising heart ratewould likely indicate irritation, in which case the entrainment routinemight be modified by reducing the entrainment to theta wave entrainmentto further induce relaxation.

Many other types and implementations of feedback are possible including,but not limited to, changing of entrainment routines based on userreactions to, or interactions with, virtual objects and virtualenvironments; user attention attributes such as the location, intensity,focus, and consistency of user attention to virtual objects and virtualenvironments; game scores and other gaming metrics; physical biofeedbacksuch as monitoring heart rate, perspiration, respiration; cognitivebiofeedback such as monitoring changes in an EEG; exercise equipmentfeedback such as treadmill speed, cycling cadence and/or power, rowingstrokes per minute and/or power. Further, entrainment routines can bechanged to use different types of stimulation (e.g., if the feedbackindicates that visual stimulation is less effective at certain points ina game, it can be supplemented with auditory or haptic feedback).Multiple stimulation devices can be used to augment or supplement thevisual stimulation including, but not limited to, haptic headbands orvest, speakers or headphones, and other stimulation devices. In thisway, the system can be programmed to automatically adapt to users basedon a variety of feedback sources.

FIG. 13 is a block diagram illustrating an exemplary system architecture1300 for a therapeutic stimulation system using virtual objects andenvironments to provide multi-modal stimulus, according to anembodiment. According to the embodiment, the system comprises a stimulusmanager 1320, a VR application 140, a therapy regimen controller 110,one or more spatial sensors 130, one or more biometric sensors 120, oneor more external transducers or stimulus devices (stimulus array 1310illustrates some non-limiting examples of such transducers and stimulusdevices), and a display 160.

The stimulus manager 1320 is the core of the system, and manages inputsfrom, and outputs to, other components of the system. It is responsiblefor selection of a therapeutic stimulation regimen (e.g., entrainment,attention sub-process training, etc.), evaluation of the user'sattention, and activation of both virtual and physical stimulationtransducers 1310. Stimulus manager 1320 may be a specifically configuredembodiment of brainwave entrainment manager 200. Stimulus manager 1320may receive various sensor data and analyze the received data todetermine (e.g., compute, calculate, infer, predict, etc.) an attentionlevel (also referred to as “attention score”) associated with a user.For example, eye tracking sensors and cameras may be used to track auser's eye movement during a regimen and a user's attention level may bedetermined based at least on the user's eye movement information.Furthermore, a user's attention score may be determined in real-time ornear real-time and used applied as feedback for the selection ofstimulus type and intensity during a user's current therapeuticstimulation regimen or a user's future regimen. The feedback based on auser's attention level or attention score results in one or more stimulibeing activated or deactivated during an ongoing regimen or applied tofuture regimens. A type of stimulus may and/or its intensity (e.g.,frequency) can be selected based on a user's prior results, which maycomprise for example, prior results from the same or similar regimen,prior results with specific types of stimulus, prior results withspecific intensities/frequencies, prior results with specificactivities, prior results with specific attention sub-process testsand/or tasks, and/or the like. For example, some individuals may be moreresponsive to certain stimuli. Further, certain stimuli can elicitspecific responses that may overcome specific attention deficits ordistractions. These stimuli may be specifically targeted by selectingappropriate transducers and stimulus devices based on feedbackcomprising at least a user's determined attention level. The selected oridentified stimulus may then be applied to an ongoing regimen inreal-time or near real-time (i.e., as the user is performing a regimen,the regimen changes based on live measured attention level information),or applied to future regimens (i.e., the system 1300 remembers theuser's attention level throughout the regimen and can make adjustmentsto future regimens based on this), or both.

The therapy regimen controller 110 is an administrative interface thatallows an administrator (e.g., a physician, therapist, masseuse, orother service provider) to select therapy regimens for application tothe user (who may be a patient, client, etc., of the administrator). Thetherapy regimen controller 110 may be used, for example, to select aregimen for therapeutic stimulation for attention bias training thatemphasizes maintenance of multiple prevalent and often co-occurringforms of psychopathology and addiction.

The biometric sensors 120 are sensors that measure a physical orphysiological characteristic of the user, such as heart rate,temperature, sweat production, brain activity (using anelectroencephalograph, or EEG), etc. Biometric sensors 120 are used toprovide feedback to the stimulus manager 1320 as to the physical orphysiological state of the user, which may be used to infer the user'smental state. For example, a biometric sensor 120 that measures theuser's heart rate may be used to infer the user's level of relaxation(or lack thereof), thus providing feedback as to the effectiveness ofalpha brainwave entrainment intended to induce relaxation.

Spatial sensors 130 are sensors that measure a user's physical locationin space or a location at which the user is focusing his or herattention. For two dimensional screens, eye movement may be tracked andthe location of the user's gaze may be calculated. In the case ofvirtual reality (VR), the user's body may be tracked, or if the user iswearing a VR headset, the orientation of the headset can be used todetect the user's head movements. Spatial sensors 130 are used to detectthe user's engagement with virtual objects and virtual environments,such that therapeutic stimulation using those objects and environmentscan be adjusted, accordingly.

The VR application 140 is used for implementation of therapeuticstimulation. While a VR application 140 is shown here, in principle anycomputer game, puzzle, display, or animation can be used, whetherinteractive or not, and whether three-dimensional or two-dimensional.The VR application 140 can be a specially-designed program intended foruse with the system, or can be an off-the-shelf game or applicationadapted for use with the system. In either case, the VR application 140will either have an interface with the brainwave entrainment manager200, or will have a stimulus manager 200 integrated into it, whereby thestimulus manager 200 is used to control therapeutic stimulation usingthe virtual objects in the VR application 140.

The stimulus array 1310 comprises a plurality of possible stimulustransducers and devices which can be implemented in various embodimentsof the system. Stimulus array 1310 can include external transducerswhich are physical stimulation transducers that may be used tocomplement therapeutic stimulation using virtual objects. A non-limitinglist of external transducers includes ultrasonic transducers 1311, aradio-frequency (“RF”) stimulator 1312, a magnetic stimulator 1313,transcranial direct current (“DC”) stimulators 1314 (e.g., electrodes),audio transducers 1315 (e.g., speakers, etc.), deep brain stimulation1316, various optical transducers 1317 (e.g., lights or LEDs), vibratoryor other pressure-producing devices, and electrical stimulators. As anexample, while brainwave entrainment is being applied visually usingvirtual objects on a screen, the brainwave entrainment may besupplemented or complemented by ultrasonic stimulation using ultrasonictransducers.

FIG. 14 is a block diagram illustrating an exemplary aspect of atherapeutic stimulation system using virtual objects and environments toprovide multi-modal stimulus, an attention evaluator 1400. According tothe aspect, attention evaluator 1400 comprises a test and resultdatabase 1410 and a scoring engine 1420. The test and result database1410 may comprise information related to one or more tests, schemes,mechanisms, and/or paradigms used to evaluate one or more sub-processesof a user's attention cognitive ability, as well as user-specificinformation related to prior regimens (e.g., prior test results, priorstimulus response results, prior regimen complete status, etc.).Attention evaluator 1400 may use information related to the stored testswhen evaluating and scoring various sub-processes of a user's attention.For example, a modified Stroop test stored in test and result database1410 may comprise information describing a range of time in which eachresponse to a specific displayed object should be received indicatingproper attention levels, and attention evaluator 1400 may retrieve theinformation describing the range of time to compare against receivedsensor data in order to evaluate the user's attention sub-process. Aplurality of attention sub-processes may be evaluated and/or trainedduring a regimen based on received sensor data 1405.

Scoring engine 1420 is configured to determine an overall attentionlevel or score 1425 for a user, wherein the overall attention score 1425represents an aggregation of attention sub-process sub-scores. In someimplementations, the sub-process sub-scores are aggregated by summingeach applicable attention sub-process sub-score. In someimplementations, each attention sub-score may be assigned a weight whichis applied to the sub-score before being aggregated into the overallattention score. According to some embodiments, weights may be set byadministrators via therapy regimen controller 110. Weights may beuser-specific and vary from user to user. In some implementations, theweighting may be based on prior and/or current user response to aregimen, stimulus, and/or activity. For example, if a user's selectiveattention sub-process is at a satisfactory level, then it may beassigned a lower weight than that of a sub-process that needs moretraining.

According to the aspect, attention evaluator 1400 is configured toreceive, retrieve, or otherwise obtain various sensor data 1405 from oneor more sensors 120, 130. The sensor data 1405 may comprise physical orphysiological characteristic information of the user such as, forexample, the user's eye movement or heart rate during a therapeuticstimulation regimen (e.g., interaction with a virtual object in avirtual space or interaction with a virtual object on a display screen,etc.). In some implementations, sensor data related to a user's eyemovement during a regimen is received and used as an input to determinethe user's attention level (i.e., attention score). In someimplementations, attention evaluator 1400 may receive, retrieve, orotherwise obtain information associated with a regimen being currentlyperformed by a user. For example, information about the types ofstimulus being applied and at what operating frequencies during aregimen. In some implementations, this information may be ascertainedfrom the selected therapeutic stimulation regimen as the regimencomprises information related to the stimulus types, stimulusfrequencies, and intensity, among other information. According to someimplementations, the regimen further comprises information related toone or more tests, exercises, and/or routines associated with measuring,monitoring, and/or training various sub-processes of attention. Sensordata collected during these attention tests may be used to determine anattention sub-score associated with an attention sub-process.

Attention evaluator 1400 is configured to determine an attention scorefor a user based on user interaction with virtual objects displayed infront of them, wherein the display of virtual object may be furtheraccompanied by one or more other stimulus types to providemulti-modality therapeutic stimulus to the user. In someimplementations, the attention score may be a composite scorerepresenting one or more attention sub-scores aggregated together. Insome implementations, each sub-score may be assigned a weight whereinthe weight is applied to the sub-score prior to being aggregated intothe overall attention score 1425. In some implementations, the weight tobe applied to a sub-score can be set or determined by an administrator(e.g., a therapist, a doctor, etc.) via therapy regimen controller 110.According to the embodiment, attention evaluator 1400 can evaluate aplurality of attention sub-processes such as, for example, attentionbias, sustained attention, alternating attention, selective attention,and divided attention; and each of these attention sub-processes may bescored based on the evaluation and applied to determine an overallattention score 1425 for the user.

An attention bias sub-score 1421 a may be determined based on userinteraction with one or more virtual objects during a regimen. Attentionbias has been conceptualized and operationalized as preferentialallocation of attention to certain (target) stimuli, relative tocompeting (neutral) stimuli. In other words, it is the tendency to payattention to some things while simultaneously ignoring others andrepresents a type of cognitive bias. There are various ways in whichattention bias can be evaluated and subsequently scored. One such methodknown to those with skill in the art is using the dot-probe task, bytiming the responses of users to threatening, neutral, and positiveimages (e.g., virtual objects) or words displayed on a screen or virtualenvironment. One other such mechanism for evaluating attention bias in auser is by applying the Stroop test in which users are asked to name thecolor of a printed word (e.g., virtual word displayed on a screen, etc.)wherein the words being shown are either emotionally negative oremotionally neutral. The Stroop test measures how long it takes a userto name the color of a word on the display. Another such method that maybe used in various implementations comprises: providing the user with atleast one sensorial stimulus (e.g., provide stimulus using one or moretransducers); measuring at least one attention allocation index bymeasuring the user's response to the stimulus; and outputting anattention bias sub-score 1421 a. In some implementations, the attentionallocation index measuring is implemented using at least one predefinedattention allocation mechanism, where the mechanism is based on at leastone of: a dot-probe task; a spatial cueing paradigm; a visual searchtask; and/or a modified Stroop task. One or more eye tracking sensor maybe used to measure the user's attention during attention bias testingduring the course of a therapeutic stimulation regimen.

A sustained attention sub-score 1421 b may be determined based on userinteraction with one or more virtual objects during a regimen. Sustainedattention relates to the ability to focus on a stimulus over an extendedperiod of time. Performing attention-related tasks in real life involvesthe need to ignore a variety of distraction and inhibit attention shiftsto irrelevant activities. Sustained attention is usually divided intovigilance and concentration. Sustained attention may be evaluated byattention evaluator 1400 during a regimen by monitoring a user's abilityto detect the appearance of a stimulus (e.g. vigilance) and the abilityto focus on the stimulus or activity in the presence of other stimuli.Once evaluated the result is the sustained attention sub-score 1421 b.

An alternating attention sub-score 1421 c may be determined based onuser interaction with one or more virtual objects during a regimen.Alternating attention relates to the ability to change focus attentionbetween two or more stimuli. Alternating attention is closely linked toworking memory and other cognitive processes. Alternating attention canbe evaluated using one or more of a variety of tests such as, forexample, the Trail Making Test Part B, the Letter-Number Sequencingsubtest from WAIS-IV, and the Symbol Digit Modalities Test. Each ofthese tests may be modified in order to be performed using virtualobjects on a display as described herein. The resulting evaluation of auser's alternating attention may be output as the alternating attentionsub-score 1421 c and used as a component when determining the user'soverall attention score 1425

A selective attention sub-score 1421 d may be determined based on userinteraction with one or more virtual objects during a regimen. Selectiveattention relates to the ability to attend to a specific stimulus oractivity in the presence of other distracting stimuli. A user'sselective attention may be evaluated by measuring the user's interactionwith a target stimulus or activity while providing extraneous anddistracting stimuli simultaneously as the target stimulus or activity.One or more eye tracking sensors may be used to measure the user'sinteraction during the regimen to evaluate the user's selectiveattention sub-processes. The result of the evaluation is the selectiveattention sub-score 1421 d which can be used as a component whendetermining the user's overall attention score 1425.

A divided attention sub-score 1421 e may be determined based on userinteraction with one or more virtual objects during a regimen. Dividedattention relates to the ability to attend different stimuli orattention at the same time. Divided attention is a type of simultaneousattention that allows individuals to process different informationsources and successfully carry out multiple tasks at a time. Thiscognitive skill is very important, as it allows people to be moreefficient in their day-to-day lives. In some implementations, thedivided attention sub-process may be evaluated by using the modifiedStroop test which is also used to test the attention bias of a user. Theresult of the evaluation is the divided attention sub-score 1421 e whichcan be used as a component when determining the user's overall attentionscore 1425.

Each of the previously described sub-processes may be scored 1421 a-eand used to determine an overall attention score 1425 or level of auser. Scoring engine 1420 may determine an overall score and send thescore to either or all of a therapeutic stimulation regimen selector201, in-game object activator 203, and/or a transducer array activator204. Regimen selector 201 may use the received attention score asfeedback when selecting a regimen or when defining a future regimen. Forexample, a stimulus is applied to a display while a user is performing aregimen, the users attention level is determined based on various sensordata collected during the regimen, and a stimulus or an stimulusfrequency may be selected based on the user's attention level. Moreover,a user's attention level may be improved by applying specific stimulus.

Detailed Description of Exemplary Aspects

FIG. 3 is a diagram of an exemplary brainwave entrainment therapy devicethat can be attached to an exercise machine for brainwave entrainmenttherapy with light and/or sound, including brainwave entrainment usingvirtual objects. In this embodiment, the brainwave entrainment devicecomprises a display 301, one or more lights 302, and one or morespeakers or headphones 303. The display 301 is used for display ofactivities designed to engage the user in games or other activitieswhile brainwave entrainment is applied using virtual objects on thedisplay. The lights 302, shown here as light bars comprising multiplelight-emitting diodes (LEDs) can be programmed to emit a supplementalvisible stimulus (e.g., flashes, on/off cycles, etc.) at frequenciesappropriate for brainwave entrainment. The speakers 303 can beprogrammed to emit a supplemental audible stimulus (e.g., rectangularwave sound pulses, sine wave sound oscillations, etc.) at frequenciesappropriate for brainwave entrainment. In some configurations, bothlight and sound may be used as stimuli, separately or in conjunctionwith brainwave entrainment using virtual objects on the display 301. Thestimuli need not be from the same source (e.g., two light sources eachat 20 Hz could be synchronized to produce a 40 Hz stimulus) or from thesame modality (e.g., a sound source at 15 Hz and a light source at 15 Hzcould be synchronized to produce a 30 Hz stimulus)

The device of this embodiment is designed such that is can be mounted onan exercise machine (that may or may not be otherwise equipped forbrainwave entrainment purposes), whereby it can be used to providebrainwave entrainment using virtual objects on the display 301,optionally with supplemental brainwave entrainment from the lights 302and/or speakers 303. The use of virtual objects with brainwaveentrainment allows for flexibility in applying brainwave entrainment.Brainwave entrainment using virtual objects provides essentiallyunlimited variability in terms of stimulator sizes, shapes, colors,movements, and allows for the use of multiple stimulatorssimultaneously, each with different characteristics. Further,gamification changes the brainwave stimulation from passive receipt oflight therapy to active engagement with the visual stimulation objects,wherein the user's brain is actively stimulated during the activity,enhancing the effectiveness of the stimulation. Further, as the user isactively engaged with the virtual objects, stimulation can be appliedbased on where the user's attention is focused. Attention-basedstimulation provides opportunities for both direct stimulation (e.g.,flashing an object at which the user is looking) and indirectstimulation (e.g., flashing an object in the user's periphery ofvision). For example, eye tracking technology can be used to determinewhere the user is looking on the screen at any given time, and objectsat which the user is looking can be used to provide visual stimulationeven if the user changes his or her attention to a different object onthe screen. In this embodiment, an infrared emitter 304 emits aninfrared light, which is reflected off the user's eye and cornea, and isreceived at an infrared-sensitive camera 305. The center of the eye istracked in relation to a reflection from the cornea (the outer surfaceof the eye). The distance and direction of the difference between thecenter of the eye and the corneal reflection can be used to calculatethe eye's position. Combined with a known distance to and size of thedisplay 301 the location at which the user is looking can be determined.The user's attention to objects on the screen can be monitored over timeto determine whether the user is remaining focused on the activity, oris getting tired and losing focus, and the determined level of userattention can be used to change the type, intensity, directness, andother characteristics of the stimulation.

Brainwave entrainment using virtual objects may be further enhanced byusing multiple objects, each capable of providing complementary types ofstimulation, and/or by intentionally directing the user's attention toobjects providing certain types of stimulation. For example, if the useris playing a first person shooter (FPS) game that involves shootingattacking aliens, the user's attention will naturally be focused onfinding attacking aliens, aiming at them, and shooting them. As eachalien will be the focus of the user's attention sequentially, the alienat which the user is currently looking may be flashed at appropriatefrequencies and in appropriate colors to provide appropriate brainwavestimulation. Simultaneously, other objects on the screen (or even thebackground) may be selected to provide a complementary visualstimulation in the periphery of the user's vision. Further, brainwaveentrainment using virtual objects may be enhanced by selecting multipletreatment modalities (e.g., light, sound, vibration, electricalstimulation) applied either simultaneously or sequentially, by varyingthe frequency or frequencies of brainwave entrainment (e.g., from about0.5 Hz to about 100 Hz), and by varying the intensity and/or scale ofthe treatment (e.g., from subtle, localized vibrational or electricalstimulation to area-wide, intense stimulation such as high-intensityroom lighting and sound).

Application of brainwave entrainment using virtual objects andgamification allows for brainwave entrainment to target certainneurological functions by enhancing and concentrating the effect of thebrainwave entrainment on the stimulated areas of the brain. As oneexample, a person with memory loss may be asked to play a memory-basedcard matching or tile matching game (mental activities which stimulatecertain portions of the brain). While the person is engaged in themental activity, brainwave entrainment is applied via the game objectson the display 301 and/or the lights 302 and/or speakers 303. As theneurological functions in the brain associated with memory are beingstimulated, the neurons in the brain associated with those functions arein an already-stimulated state, and the brainwave entrainment'sstimulation of oscillations in the electrochemical state of neurons inthose already-stimulated areas will have a more pronounced effect thanon other areas of the brain. In this way, the already-stimulated areasof the brain may experience a greater reduction in degenerativeconditions (i.e., reductions in amyloid plaques and tau phosphorylation)and greater increases in synaptic density.

FIG. 4 is a diagram of an exemplary brainwave entrainment therapy systemfor brainwave entrainment therapy that allows for multi-modal,multi-intensity therapies. The system 400 of this embodiment comprises astationary recumbent bicycle 410, and three different scales ofbrainwave entrainment stimulators: localized and/or individualstimulation transducers 420, small area stimulation transducers 430, andlarge area stimulation transducers 440.

The stationary recumbent bicycle 410 comprises a base 415, a chair back411, a seat 412, arm rests 414, a plurality of supports 413 connectingthe chair back 411 and seat 412 to the base 415, a resistance mechanism416 allowing for resistance to a pedaling motion of the user, and apedal system 417 for the user to pedal in a cycling motion. Thestationary recumbent bicycle 410 thus provides the means for the user toengage in a physical task in the case where dual task stimulation(and/or dual task assessment) is being applied.

The localized and/or individual stimulation transducers 420 of thisembodiment are a headband 421 with vibratory stimulation and hand grips422 which provide electrical stimulation. These provide localizedstimulation which can only be perceived by the user, which also makesthem individual stimulation transducers (as opposed to the other scales,which can be perceived by others, and which could be used to providebrainwave entrainment to more than one person using the sametransducer(s)). The headband 421 may produce simple vibratory (i.e.,tactile) stimulation to the head, or may be configured to producevibrations at certain locations on the head and at certain intensitiesso as to be perceptible by the middle and inner ear, which causes thestimulation to be both tactile and auditory in nature. This doublestimulation (tactile and auditory) amplifies the effect of a single typeof transducer, increasing the efficiency of brainwave entrainment fromapplications of that transducer.

The small area stimulation transducers 430 of this embodiment aredevices attached to the exercise machine 410, but not directly attachedto or in contact with the user. For example, a console comprising adisplay 432, light bars 433, and speakers 434 similar to that of thedevice of FIG. 33 may be used. The console may be attached to theexercise machine using an adjustable arm 431 that allows for optimalpositioning of the console for viewing and/or interaction by the user.Other small area stimulation transducers include a large electric motor435 with an offset weight 436 attached to the seat 412 that allows forfull-body vibratory stimulation to be applied, and a subwoofer 437 underthe chair back 411 that allows for both audible (regular sound) andinaudible (infrasound) stimulation to be applied. Small area stimulationtransducers are particularly useful in situations where direct contactwith a user is not desirable, or when multiple users will be using thedevice sequentially, or when brainwave entrainment will be applied to asmall number of users (e.g., those directly in front of the stimulationtransducers). The display 432 may be used to provide brainwaveentrainment using virtual objects in conjunction with gamification.

The large area stimulation transducers 440 of this embodiment aredevices that can be used over a large area and potentially a largenumber of persons such as a room or auditorium. In this embodiment, Thelarge area stimulation transducers are large LED light bars 442 andlarge speakers 443 attached to a wall 441 of the room in which thestimulation will be applied. The large area stimulators such as the LEDlight bars 442 and large speakers 443 on the wall 441 can be used tofully immerse the user in intense brainwave entrainment with large areasof bright light and loud, booming sounds. The immersion and intensitycan be enhanced, for example, by surrounding the user with large areastimulators on walls on all sides (and possibly ceilings and floors)covering the user's entire visual area, so that the user receives visualstimulation no matter in which direction the user looks an auditorystimulation no matter where the user is located. Higher immersion andintensity may provide greater beneficial effects from brainwaveentrainment.

It is important to note that any type of transducer can be applied atany scale. For example, light stimulation can be configured such that itis seen only by one person (e.g., in glasses or goggles), or is seen bya small number of persons (e.g., a single LED light bar), or is seen bymany people (e.g. room lights, stadium lights, etc.). Further, theintensity of stimulation can be largely varied separately from the scaleof stimulation. However, depending on the circumstances and application,brainwave entrainment at certain scales and/or intensities may be moreuseful or effective than at others.

The different scales of stimulation transducers allow for a choice ofthe level of immersion the user experiences with respect to thebrainwave entrainment, and to some degree, the level of intensity of thebrainwave entrainment. Immersion is the quality of being surrounded byor absorbed in an experience. Intensity is the magnitude of theexperience. They are separate qualities (e.g., a localized electricstimulation can be intense, but not immersive), but there can be anincrease in intensity with an increase in scale (for example, if lightstimulation comes from all directions, it will tend to be both moreimmersive and more intense, although the intensity of the lights can bereduced to offset this tendency). For example, a localized, subtleelectrical stimulation through electrically-conducting hand grips 422provides minimal immersion of the user in the brainwave entrainment.This may be useful, for example, where intense concentration on the dualtask stimulation is necessary. Small area stimulation transducers suchas the LED light bars 433 on the screen console are useful for mid-levelimmersion and mid-level intensity of brainwave entrainment. The LEDlight bars 433 cover a small, but significant, area of the user's view,and the speakers 44 are large enough to provide a substantial auditorystimulus. The large area stimulators such as the LED light bars 442 andlarge speakers 443 on the wall 441 can be used to fully immerse the userin intense brainwave entrainment with large areas of bright light andloud, booming sounds. The immersion and intensity can be enhanced, forexample, by surrounding the user with large area stimulators on walls onall sides (and possibly ceilings and floors) covering the user's entirevisual area, so that the user receives visual stimulation no matter inwhich direction the user looks an auditory stimulation no matter wherethe user is located. Higher immersion and intensity may provide greaterbeneficial effects from brainwave entrainment.

Further, it is important to note that the modalities (types ofstimulation), scales, and intensities allows for tremendous flexibilityin selecting suitable therapies regimens for different situations. Forhigh-immersion scenarios (e.g., maximum brainwave entrainment with fewercognitive demands such as listening to music), multiple modalities,scales, and intensities may be used at the same time. For example, whilea user is listening to classical music, localized electrical stimulationmay be applied to the wrist, small area visual stimulation may beapplied using a single LED light bar, and large area tactile stimulationmay be applied using subwoofers which produce sounds (infrasounds) whichare inaudible to the human ear but can be perceived through the sense oftouch (e.g., as oscillating pressure on the torso).

Further, modalities can be chosen to either amplify certain tasks oractivities or to supplement them. For amplification, treatmentmodalities are chosen to include those corresponding to a given task oractivity in gamification. As an example, if a user is assigned a gameactivity wherein the user must follow a moving object on the displaywith his or her eyes, the object can be flashed at 40 Hz for gammaentrainment therapy. As the user is already focused on the object, theuser is focusing more intensely on visual activities (and the brainareas and functions associated with visual activities are stimulated),enhancing the effect of the visual gamma entrainment modality. Forsupplementation, treatment modalities are chosen to exclude thosecorresponding to a gamification task. As an example, if game activityassigned to a user is identifying songbirds presented on the display,flashing the birds at 40 Hz (or otherwise changing their colors orvisual appearance) may interfere with the identification process. Insuch circumstances, a non-conflicting modality may be chosen such asflashing of background objects or supplementation with audibleentrainment.

FIGS. 5A & 5B are a flow diagram showing an algorithm for selection ofmodalities and routines for brainwave entrainment and application ofbrainwave entrainment using a virtual environment using eye tracking andbiometric feedback to select virtual objects and entrainment routines.As a first step, a therapy regimen is received 501 The therapy regimenmay be received from any source providing instructions for brainwaveentrainment, such as a database, an administrator (e.g., a physician,therapist, masseuse, or other service provider) for application to auser (who may be a patient, client, etc., of the administrator), or fromthe user himself or herself. An example therapy regimen would be aregimen for brainwave entrainment that emphasizes alpha wave stimulationto induce relaxation in an overstimulated user.

A suitable VR application or other gamification application is thenchosen 502, which ideally should be consistent in content with thenature of the therapy regimen chosen. For example, if the therapyregimen is a regimen for brainwave entrainment that emphasizes alphawave stimulation to induce relaxation in an overstimulated user, a VRapplication might be chosen that involves causal cycling along a forestpath. If a more stimulating therapy regimen is chosen, for examplesomething involving intense concentration and gamma wave therapy, afirst person shooter might be chosen.

Based on the therapy regimen and VR application chosen, an entrainmentroutine is selected 503. For example, if the therapy regimen specifiesthat the overall brainwave entrainment goal is relaxation, theentrainment routine selected 503 may use alpha wave entrainment as theprimary entrainment therapy, and may choose to apply alpha waveentrainment to a background virtual object (e.g., the sky or trees inthe background of the casual cycling along the forest path), as flashingof background objects will be less intrusive (and possibly morerelaxing) to the user than flashing of objects to which the user'sattention is directed (e.g., the path or direction of the virtualbicycle). Selection of the entrainment routine 503 may further involveselecting amplification or supplementation 504 as appropriate for thecircumstances, choosing appropriate treatment modalities (e.g., lighttherapy, sound therapy, vibrational therapy, electrical therapy, orcombinations of such modalities) either for amplification 505(treatments including those corresponding to the tasks, activities, orneurological function) or for supplementation 506 (treatments includingthose corresponding to the tasks, activities, or neurological function),and selecting a stimulation scale and intensity 507 for each modalityappropriate for the treatment goals. In this example, three modalitiesare shown with different scales and intensities, localized hapticstimulation at a light intensity 507 a, large area visual stimulation ata moderate intensity 507 b, and small area auditory stimulation at amoderately intense intensity 507 c. Brainwave entrainment is thenapplied using the chosen regimen, providing targeted treatment ofparticular areas of the brain and/or particular neurological functionsvia stimulation of those areas or functions using dual task stimulation.

At this point, a camera may be used to track the user's eye movements508 to determine where the user is looking on the screen at a givenmoment 509. Based on the above inputs, appropriate virtual objects arechosen to apply brainwave entrainment by modifying virtual objects onthe screen 510, which modification may take any number of forms (e.g.,objects may be flashed at specific frequencies, the color of objects maybe changed at specific frequencies, the size of objects may be changedat specific frequencies, objects may be rotated at specific frequencies,etc.). Any change to a virtual object that is perceptible to a user andcan be applied at a repeating frequency (i.e., oscillating frequency)may be used to apply brainwave entrainment. Brainwave entrainment isapplied using the virtual objects, optionally supplemented withentrainment from external transducers 511.

Input from biometric feedback (e.g., the user's heart rate) is received512 and evaluated to determine whether the selected entrainment routineis having the desired effect (e.g., a lowering heart rate may be used toinfer relaxation), and to change the entrainment routine, accordingly513. For example, a lowering heart rate during alpha wave entrainmentwould likely indicate relaxation, in which case the entrainment routinewould remain unmodified, but a rising heart rate would likely indicateirritation, in which case the entrainment routine might be modified byreducing the entrainment to theta wave entrainment to further inducerelaxation. The process of tracking the user's attention and applyingappropriate modifications to brainwave entrainment is repeated from step508 until the therapy session ends.

FIG. 6 is a diagram showing explaining the use of duty cycles and pulsewidth modulations in applying brainwave entrainment. Here, threeexamples 610, 620, and 630 of duty cycles/pulse width modulation areshown. The frequency of stimulation 602 in all three examples is 40 Hz(40 cycles per second), and the wave form of each example is arectangular wave (i.e., instantaneous or near-instantaneous changesbetween on and off states). Three periods 601 a-c of the stimulation atthe 40 Hz frequency 602 are shown, each period corresponding to one fullon/off cycle lasting 1/40 of one second. In Example 1 610, a duty cycleof 50% is shown in which the stimulation is in an on state 611 for 50%of the period and in an off state 612 for 50% of the period. For a 40 Hzfrequency as shown here, this corresponds to a pulse width of 1/80^(th)of a second, wherein the stimulation is in an on state 611 for 1/80^(th)of a second and in an off state 612 for 1/80^(th) of a second. InExample 2 620, a duty cycle of 25% is shown in which the stimulation isin an on state 621 for 25% of the period and in an off state 622 for 75%of the period. For a 40 Hz frequency as shown here, this corresponds toa pulse width of 1/160^(th) of a second, wherein the stimulation is inan on state 621 for 1/160^(th) of a second and in an off state 622 for3/160^(th) of a second. In Example 3 630, a duty cycle of 75% is shownin which the stimulation is in an on state 631 for 75% of the period andin an off state 632 for 25% of the period. For a 40 Hz frequency asshown here, this corresponds to a pulse width of 3/160^(th) of a second,wherein the stimulation is in an on state 631 for 3/160^(th) of a secondand in an off state 632 for 1/160^(th) of a second.

FIGS. 7-9 (PRIOR ART) explain the application of eye tracking technologyas a means of determining where a user is looking. In one form of eyetracking technology, an infrared emitter 720 emits an infrared light721, which is reflected off the user's eye 701 and cornea, and isreceived 731 at an infrared-sensitive camera 730. The image of the ofthe user's eye appears to the camera substantially as shown in FIG. 9 ,wherein the sclera (the white part of the eye) 901, the iris (thecolored part of the eye) 902, and the pupil (the opening in the eye) 903are visible. The center of the eye 910 is tracked, as shown by a firstset of crosshairs 911, in relation to a reflection from the cornea (theouter surface of the eye) 920, as shown by a second set of crosshairs921. The distance and direction of the difference between the center ofthe eye and the corneal reflection can be used to calculate the eye'sposition. Combined with a known distance to and size of a display, 740,the location at which the user is looking 702 can be determined. FIG. 8shows the same application of eye tracking technology, but inside a VRheadset 840. In FIG. 8 , an infrared emitter 820 emits an infrared light821, which is reflected off the user's eye 801 and cornea, and isreceived 832 at an infrared-sensitive camera 830. The distance anddirection of the difference between the center of the eye and thecorneal reflection can be used to calculate the eye's position. Combinedwith a known distance to and size of a display, 841, the location atwhich the user is looking 802 can be determined.

FIG. 10 is a diagram showing an embodiment in which on-screen virtualobjects on a display are used to apply brainwave entrainment. In thisexample, brainwave entrainment is implemented using a display 1010, suchas a television computer monitor, or tablet-based device, comprising ascreen 1011 and in some configurations, built in speakers 1031 a,b. Inthis embodiment, the screen 1011 is used to provide visual brainwaveentrainment, either by flashing the background of the screen 1011 or oneor more on-screen virtual objects 1020. This embodiment enables theprovision of brainwave entrainment without the use of (or in additionto) external devices such as lights and speakers. In this example, fiveon-screen virtual objects 1020 are shown 1021-1025, each comprising adifferent shape and each moving independently on the screen 1011 asindicated by the dashed and dotted “movement shadows” associated witheach on-screen virtual objects 1020. The on-screen virtual objects 1020are generic shapes in this diagram, but may represent any type ofon-screen element whether static or movable, permanent or transient.Depending on the configuration, the on-screen element may be any shapeor color displayable on a screen, such as game elements, puzzleelements, background elements, regular or irregular portions of thescreen. Many possible applications of this embodiment are possible. Thebuilt-in speakers, if any, may be used to provide auditory brainwaveentrainment in addition to the visual on-screen brainwave entrainment.

For example, when paired with a camera and eye-tracking software, theon-screen virtual objects 1020 might represent an eye musclestrengthening exercise combined with brainwave entrainment, wherein theuser is asked to find a target on-screen virtual object with aparticular shape and follow the shape with his or her eyes. At the sametime the target virtual object may flash a particular color at aselected brainwave entrainment frequency, with the color changing as theuser's eyes either follow the target on-screen virtual object or strayfrom it. The target on-screen virtual object may, for example, be apleasant light-blue color while the user's eyes are following it, andchange to a bright red to re-attract the user if the user's eyes startfollowing a different on-screen element.

In this embodiment, a clip-on eye-tracking unit 1040 may be attached tothe display 1010 using plastic (or other material) clips 1044. Theclip-on eye-tracking unit 1040 comprises a housing 1041, an infraredemitter 1042 which emits an infrared light that is reflected off theuser's eye and cornea, and is received at an infrared-sensitive camera1043, and clips 1044 which may be used to attach the clip-oneye-tracking unit 1040 to a display 1010. The center of the eye istracked in relation to a reflection from the cornea (the outer surfaceof the eye). The distance and direction of the difference between thecenter of the eye and the corneal reflection can be used to calculatethe eye's position. Combined with a known distance to and size of thedisplay 1010 the location at which the user is looking can bedetermined.

In another use case, the on-screen virtual objects 1020 may represent apuzzle or game, and the brainwave entrainment may be provided by simplyflashing the screen background 1012 at a selected brainwave entrainmentfrequency.

This example may be extended to virtual reality applications, whereinbrainwave entrainment is provided by flashing in-game elements withinthe virtual reality environment.

FIG. 11 is a diagram showing an exemplary virtual reality environment inwhich virtual objects may be used as visual stimulation transducers. Thevirtual reality environment show in this diagram depicts a quiet scenefrom a first person perspective, and would be suitable for brainwaveentrainment related to theta or alpha wave entrainment (for example, tofacilitate relaxation, creativity, exploration, and contemplation). Theenvironment comprises a room with a floor 1110, a ceiling 1112, andthree visible walls 1111 a-c. In the ceiling are four recessed lights1113. On the left wall 1111 a is a flat-screen television 1123 showingan outdoor scene 1124 involving mountains, trees, and lightning. On theright wall 1111 c is a door 1114. On the back wall is a window to theoutside 1115 in which the sun can be seen 1130. In the corner of theroom is a potted plant 1122, and next to the back wall 1111 b is a table1120 on which is standing a lamp 1121. Each and every virtual objectnamed above can be used to provide brainwave entrainment. For example,any one or all of the virtual lighting objects, the lamp 1121, thetelevision, 1124, the ceiling lights 1113, and the sun 1130 could beflashed or changed in intensity at the selected brainwave entrainmentfrequency. Even objects not associated with lighting, such as the walls1111 a-c, ceiling 1112, floor 1110, or door 1114, could be flashed orchanged. If appropriate to the therapy regimen selected, exploration andcuriosity could be encouraged by flashing certain objects (e.g., thetelevision 1124, the potted plant 1122, the table 1120, the door 1114)as the user investigates or interacts with them. With some additions, ascene such as the one depicted here could be used to perform brainwaveentrainment in a mystery or other storyline. Other modalities ofbrainwave entrainment such as sound and haptic feedback may be appliedsimultaneously with the visual stimulation. As more fully describedabove, these other modalities may be applied using either the same ordifferent brainwave entrainment frequencies. As a non-limiting example,if a user in the virtual reality environment switches on the lamp 1121,not only might the lamp 1121 flash or change color at a brainwaveentrainment frequency as a form of visual stimulation, an audible tonemight be generated corresponding to the lamp flickering at the sameentrainment frequency, and haptic feedback in the form of vibration of agame controller might also be applied. In some applications, for examplein virtual environments comprising a darkened environment such as a roomwith the lights turned off, the visual stimulation may not be used, butthe auditory and/or haptic stimulation modalities may continue to beapplied.

FIG. 12 is a diagram showing exemplary gamification of brainwaveentrainment in which in-game objects and elements are used as visualstimulation transducers in conjunction with gameplay activities. Thegameplay example shown here depicts a first person shooter (FPS)involving shooting of attacking aliens, and would be suitable forbrainwave entrainment related to beta or gamma wave entrainment (forexample, to facilitate concentration, planning, or problem-solving). Theenvironment comprises a laser gun 1220 controllable by the user, aspaceship 1212, a space background 1213 comprising stars 1210, and aplurality of attacking aliens 2111. The laser gun 1220 is shown herewith a laser flash 1221, the resulting laser beam 1222, and its impact1223 on one of the attacking aliens 1211. Each and every virtual objectnamed above can be used to provide brainwave entrainment. For example,aliens 1211 may be flashed or changed as the user's attention focuses onthem. The laser flash 1221, laser beam 1222, and impact 1223 can all beused to provide bright visual stimulation at an appropriate frequencyduring game play. Even the background 1213 and stars 1210 could bechanged in color or brightness at an appropriate frequency.

In some embodiments, virtual reality environments and games could beused to provide entrainment opposite of the common expectation. Forexample, in the calm room shown in FIG. 11 , gamma wave brainwaveentrainment associated with concentration and planning could be appliedto increase the user's awareness when in calm or innocuous-lookingenvironments. Similarly, while playing an intense FPS such as that shownin FIG. 12 , theta or alpha wave entrainment could be applied to calmthe user during otherwise-intense game play. In a related use case wherea user is addicted to the adrenalin received from intense game play,theta or alpha brainwave entrainment could be used to reduce theplayer's addition to games by calming the player during intense gameplay, reducing the adrenalin rush from playing highly-immersive,fast-action games with intense themes.

Other modalities of brainwave entrainment such as sound and hapticfeedback may be applied simultaneously with the visual stimulation. Asmore fully described above, these other modalities may be applied usingeither the same or different brainwave entrainment frequencies. As anon-limiting example, when the user in the virtual reality environmentshoots the alien 1211, not only might the impact 1223 provide visualbrainwave entrainment, but an audible tone might be generatedcorresponding to the flashing or color changing of the impact 1223 atthe same entrainment frequency, and haptic feedback in the form ofvibration of a game controller might also be applied. In someapplications, for example in virtual environments comprising a darkenedenvironment, the visual stimulation may not be used, but the auditoryand/or haptic stimulation modalities may continue to be applied.

FIG. 15 is a flow diagram illustrating an exemplary method 1500 forevaluating and scoring an attention level of user, according to anembodiment. According to the embodiment, the process begins at step 1502when a therapeutic stimulation regimen is selected by stimulus manager1320. At step 1504 stimulus manager 1320 instructs or activates one ormore transducers or stimulus devices to apply stimulus based on theselected therapeutic stimulation regimen. The stimulus may be applied toone or more virtual objects on a display or in a virtual environment.The stimulus may be multi-modal wherein a stimulus is applied virtually(e.g., flashing virtual objects) and external stimuli may also beapplied (e.g., ultrasonic transducers, haptic sensors, etc.)simultaneously. In some implementations, the selected regimen maycomprise one or more attention sub-process tests, schemes, mechanisms,and/or paradigms used to measure and evaluate one or more attentionsub-processes. In such implementations, stimulus manager 1320 can applystimulus to conduct the one or more tests according to the selectedregimen. Responsive to applied stimulus, stimulus manager 1320 mayreceive, retrieve, or otherwise obtain data associated with a userinteraction with a virtual object at step 1506. In some implementations,user interaction data may be received from one or more sensors 120, 130which measure a user's physical and/or physiological characteristics.For example, eye tracking sensors may be used to collect data related toa user's eye movement during a regimen and the collected data can beused to evaluate one or more of the user's attention sub-processes. Atstep 1508, attention evaluator 1400 evaluates and scores one or moreattention sub-processes based on received user interaction data. At step1510, the one or more attention sub-scores 1421 a-e may be used todetermine a user's overall attention score or level. It may be the casethat not every attention sub-process can be evaluated and subsequentlyscored. In such cases, attention evaluator 1400 is able to still make anoverall attention score determination. The user's overall attentionscore may be a composite score computed by aggregating each availableattention sub-score, which may or may not be weighted. At step 1512,stimulus manager 1320 can apply the user's overall attention score asfeedback to a regimen currently being performed by a user, or applied tofuture regimens. The attention score may be used to activate ordeactivate one or more stimulation transducers or devices as well as toadjust the operating frequency of such transducers and devices.

Hardware Architecture

Generally, the techniques disclosed herein may be implemented onhardware or a combination of software and hardware. For example, theymay be implemented in an operating system kernel, in a separate userprocess, in a library package bound into network applications, on aspecially constructed machine, on an application-specific integratedcircuit (ASIC), or on a network interface card.

Software/hardware hybrid implementations of at least some of theembodiments disclosed herein may be implemented on a programmablenetwork-resident machine (which should be understood to includeintermittently connected network-aware machines) selectively activatedor reconfigured by a computer program stored in memory. Such networkdevices may have multiple network interfaces that may be configured ordesigned to utilize different types of network communication protocols.A general architecture for some of these machines may be describedherein in order to illustrate one or more exemplary means by which agiven unit of functionality may be implemented. According to specificembodiments, at least some of the features or functionalities of thevarious embodiments disclosed herein may be implemented on one or moregeneral-purpose computers associated with one or more networks, such asfor example an end-user computer system, a client computer, a networkserver or other server system, a mobile computing device (e.g., tabletcomputing device, mobile phone, smartphone, laptop, or other appropriatecomputing device), a consumer electronic device, a music player, or anyother suitable electronic device, router, switch, or other suitabledevice, or any combination thereof. In at least some embodiments, atleast some of the features or functionalities of the various embodimentsdisclosed herein may be implemented in one or more virtualized computingenvironments (e.g., network computing clouds, virtual machines hosted onone or more physical computing machines, or other appropriate virtualenvironments).

Referring now to FIG. 16 , there is shown a block diagram depicting anexemplary computing device 10 suitable for implementing at least aportion of the features or functionalities disclosed herein. Computingdevice 10 may be, for example, any one of the computing machines listedin the previous paragraph, or indeed any other electronic device capableof executing software- or hardware-based instructions according to oneor more programs stored in memory. Computing device 10 may be configuredto communicate with a plurality of other computing devices, such asclients or servers, over communications networks such as a wide areanetwork a metropolitan area network, a local area network, a wirelessnetwork, the Internet, or any other network, using known protocols forsuch communication, whether wireless or wired.

In one embodiment, computing device 10 includes one or more centralprocessing units (CPU) 12, one or more interfaces 15, and one or morebusses 14 (such as a peripheral component interconnect (PCI) bus). Whenacting under the control of appropriate software or firmware, CPU 12 maybe responsible for implementing specific functions associated with thefunctions of a specifically configured computing device or machine. Forexample, in at least one embodiment, a computing device 10 may beconfigured or designed to function as a server system utilizing CPU 12,local memory 11 and/or remote memory 16, and interface(s) 15. In atleast one embodiment, CPU 12 may be caused to perform one or more of thedifferent types of functions and/or operations under the control ofsoftware modules or components, which for example, may include anoperating system and any appropriate applications software, drivers, andthe like.

CPU 12 may include one or more processors 13 such as, for example, aprocessor from one of the Intel, ARM, Qualcomm, and AMD families ofmicroprocessors. In some embodiments, processors 13 may includespecially designed hardware such as application-specific integratedcircuits (ASICs), electrically erasable programmable read-only memories(EEPROMs), field-programmable gate arrays (FPGAs), and so forth, forcontrolling operations of computing device 10. In a specific embodiment,a local memory 11 (such as non-volatile random access memory (RAM)and/or read-only memory (ROM), including for example one or more levelsof cached memory) may also form part of CPU 12. However, there are manydifferent ways in which memory may be coupled to system 10. Memory 11may be used for a variety of purposes such as, for example, cachingand/or storing data, programming instructions, and the like. It shouldbe further appreciated that CPU 12 may be one of a variety ofsystem-on-a-chip (SOC) type hardware that may include additionalhardware such as memory or graphics processing chips, such as a QUALCOMMSNAPDRAGON™ or SAMSUNG EXYNOS™ CPU as are becoming increasingly commonin the art, such as for use in mobile devices or integrated devices.

As used herein, the term “processor” is not limited merely to thoseintegrated circuits referred to in the art as a processor, a mobileprocessor, or a microprocessor, but broadly refers to a microcontroller,a microcomputer, a programmable logic controller, anapplication-specific integrated circuit, and any other programmablecircuit.

In one embodiment, interfaces 15 are provided as network interface cards(NICs). Generally, NICs control the sending and receiving of datapackets over a computer network; other types of interfaces 15 may forexample support other peripherals used with computing device 10. Amongthe interfaces that may be provided are Ethernet interfaces, frame relayinterfaces, cable interfaces, DSL interfaces, token ring interfaces,graphics interfaces, and the like. In addition, various types ofinterfaces may be provided such as, for example, universal serial bus(USB), Serial, Ethernet, FIREWIRE™, THUNDERBOLT™, PCI, parallel, radiofrequency (RF), BLUETOOTH™, near-field communications (e.g., usingnear-field magnetics), 802.11 (WiFi), frame relay, TCP/IP, ISDN, fastEthernet interfaces, Gigabit Ethernet interfaces, Serial ATA (SATA) orexternal SATA (ESATA) interfaces, high-definition multimedia interface(HDMI), digital visual interface (DVI), analog or digital audiointerfaces, asynchronous transfer mode (ATM) interfaces, high-speedserial interface (HSSI) interfaces, Point of Sale (POS) interfaces,fiber data distributed interfaces (FDDIs), and the like. Generally, suchinterfaces 15 may include physical ports appropriate for communicationwith appropriate media. In some cases, they may also include anindependent processor (such as a dedicated audio or video processor, asis common in the art for high-fidelity A/V hardware interfaces) and, insome instances, volatile and/or non-volatile memory (e.g., RAM).

Although the system shown in FIG. 16 illustrates one specificarchitecture for a computing device 10 for implementing one or more ofthe inventions described herein, it is by no means the only devicearchitecture on which at least a portion of the features and techniquesdescribed herein may be implemented. For example, architectures havingone or any number of processors 13 may be used, and such processors 13may be present in a single device or distributed among any number ofdevices. In one embodiment, a single processor 13 handles communicationsas well as routing computations, while in other embodiments a separatededicated communications processor may be provided. In variousembodiments, different types of features or functionalities may beimplemented in a system according to the invention that includes aclient device (such as a tablet device or smartphone running clientsoftware) and server systems (such as a server system described in moredetail below).

Regardless of network device configuration, the system of the presentinvention may employ one or more memories or memory modules (such as,for example, remote memory block 16 and local memory 11) configured tostore data, program instructions for the general-purpose networkoperations, or other information relating to the functionality of theembodiments described herein (or any combinations of the above). Programinstructions may control execution of or comprise an operating systemand/or one or more applications, for example. Memory 16 or memories 11,16 may also be configured to store data structures, configuration data,encryption data, historical system operations information, or any otherspecific or generic non-program information described herein.

Because such information and program instructions may be employed toimplement one or more systems or methods described herein, at least somenetwork device embodiments may include nontransitory machine-readablestorage media, which, for example, may be configured or designed tostore program instructions, state information, and the like forperforming various operations described herein. Examples of suchnontransitory machine-readable storage media include, but are notlimited to, magnetic media such as hard disks, floppy disks, andmagnetic tape; optical media such as CD-ROM disks; magneto-optical mediasuch as optical disks, and hardware devices that are speciallyconfigured to store and perform program instructions, such as read-onlymemory devices (ROM), flash memory (as is common in mobile devices andintegrated systems), solid state drives (SSD) and “hybrid SSD” storagedrives that may combine physical components of solid state and hard diskdrives in a single hardware device (as are becoming increasingly commonin the art with regard to personal computers), memristor memory, randomaccess memory (RAM), and the like. It should be appreciated that suchstorage means may be integral and non-removable (such as RAM hardwaremodules that may be soldered onto a motherboard or otherwise integratedinto an electronic device), or they may be removable such as swappableflash memory modules (such as “thumb drives” or other removable mediadesigned for rapidly exchanging physical storage devices),“hot-swappable” hard disk drives or solid state drives, removableoptical storage discs, or other such removable media, and that suchintegral and removable storage media may be utilized interchangeably.Examples of program instructions include both object code, such as maybe produced by a compiler, machine code, such as may be produced by anassembler or a linker, byte code, such as may be generated by forexample a JAVA™ compiler and may be executed using a Java virtualmachine or equivalent, or files containing higher level code that may beexecuted by the computer using an interpreter (for example, scriptswritten in Python, Perl, Ruby, Groovy, or any other scripting language).

In some embodiments, systems according to the present invention may beimplemented on a standalone computing system. Referring now to FIG. 17 ,there is shown a block diagram depicting a typical exemplaryarchitecture of one or more embodiments or components thereof on astandalone computing system. Computing device 20 includes processors 21that may run software that carry out one or more functions orapplications of embodiments of the invention, such as for example aclient application 24. Processors 21 may carry out computinginstructions under control of an operating system 22 such as, forexample, a version of MICROSOFT WINDOWS™ operating system, APPLE MACOS™or iOS™ operating systems, some variety of the Linux operating system,ANDROID™ operating system, or the like. In many cases, one or moreshared services 23 may be operable in system 20, and may be useful forproviding common services to client applications 24. Services 23 may forexample be WINDOWS™ services, user-space common services in a Linuxenvironment, or any other type of common service architecture used withoperating system 21. Input devices 28 may be of any type suitable forreceiving user input, including for example a keyboard, touchscreen,microphone (for example, for voice input), mouse, touchpad, trackball,or any combination thereof. Output devices 27 may be of any typesuitable for providing output to one or more users, whether remote orlocal to system 20, and may include for example one or more screens forvisual output, speakers, printers, or any combination thereof. Memory 25may be random-access memory having any structure and architecture knownin the art, for use by processors 21, for example to run software.Storage devices 26 may be any magnetic, optical, mechanical, memristor,or electrical storage device for storage of data in digital form (suchas those described above, referring to FIG. 16 ). Examples of storagedevices 26 include flash memory, magnetic hard drive, CD-ROM, and/or thelike.

In some embodiments, systems of the present invention may be implementedon a distributed computing network, such as one having any number ofclients and/or servers. Referring now to FIG. 18 , there is shown ablock diagram depicting an exemplary architecture 30 for implementing atleast a portion of a system according to an embodiment of the inventionon a distributed computing network. According to the embodiment, anynumber of clients 33 may be provided. Each client 33 may run softwarefor implementing client-side portions of the present invention; clientsmay comprise a system 20 such as that illustrated in FIG. 17 . Inaddition, any number of servers 32 may be provided for handling requestsreceived from one or more clients 33. Clients 33 and servers 32 maycommunicate with one another via one or more electronic networks 31,which may be in various embodiments any of the Internet, a wide areanetwork, a mobile telephony network (such as CDMA or GSM cellularnetworks), a wireless network (such as WiFi, WiMAX, LTE, and so forth),or a local area network (or indeed any network topology known in theart; the invention does not prefer any one network topology over anyother). Networks 31 may be implemented using any known networkprotocols, including for example wired and/or wireless protocols.

In addition, in some embodiments, servers 32 may call external services37 when needed to obtain additional information, or to refer toadditional data concerning a particular call. Communications withexternal services 37 may take place, for example, via one or morenetworks 31. In various embodiments, external services 37 may compriseweb-enabled services or functionality related to or installed on thehardware device itself. For example, in an embodiment where clientapplications 24 are implemented on a smartphone or other electronicdevice, client applications 24 may obtain information stored in a serversystem 32 in the cloud or on an external service 37 deployed on one ormore of a particular enterprise's or user's premises.

In some embodiments of the invention, clients 33 or servers 32 (or both)may make use of one or more specialized services or appliances that maybe deployed locally or remotely across one or more networks 31. Forexample, one or more databases 34 may be used or referred to by one ormore embodiments of the invention. It should be understood by one havingordinary skill in the art that databases 34 may be arranged in a widevariety of architectures and using a wide variety of data access andmanipulation means. For example, in various embodiments one or moredatabases 34 may comprise a relational database system using astructured query language (SQL), while others may comprise analternative data storage technology such as those referred to in the artas “NoSQL” (for example, HADOOP CASSANDRA™, GOOGLE BIGTABLE™, and soforth). In some embodiments, variant database architectures such ascolumn-oriented databases, in-memory databases, clustered databases,distributed databases, or even flat file data repositories may be usedaccording to the invention. It will be appreciated by one havingordinary skill in the art that any combination of known or futuredatabase technologies may be used as appropriate, unless a specificdatabase technology or a specific arrangement of components is specifiedfor a particular embodiment herein. Moreover, it should be appreciatedthat the term “database” as used herein may refer to a physical databasemachine, a cluster of machines acting as a single database system, or alogical database within an overall database management system. Unless aspecific meaning is specified for a given use of the term “database”, itshould be construed to mean any of these senses of the word, all ofwhich are understood as a plain meaning of the term “database” by thosehaving ordinary skill in the art.

Similarly, most embodiments of the invention may make use of one or moresecurity systems 36 and configuration systems 35. Security andconfiguration management are common information technology (IT) and webfunctions, and some amount of each are generally associated with any ITor web systems. It should be understood by one having ordinary skill inthe art that any configuration or security subsystems known in the artnow or in the future may be used in conjunction with embodiments of theinvention without limitation, unless a specific security 36 orconfiguration system 35 or approach is specifically required by thedescription of any specific embodiment.

FIG. 19 shows an exemplary overview of a computer system 40 as may beused in any of the various locations throughout the system. It isexemplary of any computer that may execute code to process data. Variousmodifications and changes may be made to computer system 40 withoutdeparting from the broader scope of the system and method disclosedherein. Central processor unit (CPU) 41 is connected to bus 42, to whichbus is also connected memory 43, nonvolatile memory 44, display 47,input/output (I/O) unit 48, and network interface card (NIC) 53. I/Ounit 48 may, typically, be connected to peripherals such as a keyboard49, pointing device 50, hard disk 52, real-time clock 51, a camera 57,and other peripheral devices. NIC 53 connects to network 54, which maybe the Internet or a local network, which local network may or may nothave connections to the Internet. The system may be connected to othercomputing devices through the network via a router 55, wireless localarea network 56, or any other network connection. Also shown as part ofsystem 40 is power supply unit 45 connected, in this example, to a mainalternating current (AC) supply 46. Not shown are batteries that couldbe present, and many other devices and modifications that are well knownbut are not applicable to the specific novel functions of the currentsystem and method disclosed herein. It should be appreciated that someor all components illustrated may be combined, such as in variousintegrated applications, for example Qualcomm or Samsungsystem-on-a-chip (SOC) devices, or whenever it may be appropriate tocombine multiple capabilities or functions into a single hardware device(for instance, in mobile devices such as smartphones, video gameconsoles, in-vehicle computer systems such as navigation or multimediasystems in automobiles, or other integrated hardware devices).

In various embodiments, functionality for implementing systems ormethods of the present invention may be distributed among any number ofclient and/or server components. For example, various software modulesmay be implemented for performing various functions in connection withthe present invention, and such modules may be variously implemented torun on server and/or client components.

The skilled person will be aware of a range of possible modifications ofthe various embodiments described above. Accordingly, the presentinvention is defined by the claims and their equivalents.

What is claimed is:
 1. A system for therapeutic stimulation usingvirtual objects, comprising: a computing device comprising a memory anda processor; a display; a stimulus manager comprising a first pluralityof programming instructions stored in the memory and operating on theprocessor, wherein the first plurality of programming instructions, whenoperating on the processor, causes the computing device to: receive aregimen for therapeutic stimulation; select one or more stimulationfrequencies based on the regimen; receive data from a virtual reality(VR) application, the data comprising a location of a virtual objectdisplayed on the display; based on the regimen, instruct the VRapplication to change a visual state of the virtual object on thedisplay at the selected stimulation frequencies; and change one or moreof the stimulation frequencies based on feedback, wherein the feedbackcomprises determination of a user's attention based on the user'sinteraction with the virtual object.
 2. The system of claim 1, furthercomprising the VR application comprising a second plurality ofprogramming instructions stored in the memory and operating on theprocessor, wherein the second plurality of programming instructions,when operating on the processor, causes the computing device to: operatea virtual environment on the computing device, the virtual environmentcomprising the virtual object displayed on the display; receive theinstruction to change the visual state of the virtual object; and changethe visual state of the virtual object on the display at the selectedstimulation frequencies.
 3. The system of claim 1, wherein the user'sinteraction with the virtual object comprises user eye movement datacaptured using one or more eye tracking sensors.
 4. The system of claim3, wherein the regimen comprises one or more schemes for evaluating oneor more attention sub-processes.
 5. The system of claim 4, wherein theattention sub-processes comprise at least attention bias, selectiveattention, divided attention, sustained attention, and alternatingattention.
 6. The system of claim 5, wherein the stimulus manager isfurther configured to use the user eye movement data to determine anattention sub-score for each applicable attention sub-process.
 7. Thesystem of claim 6, wherein the determination of the user's attention isbased on an aggregation of each of the attention sub-scores.
 8. Thesystem of claim 6, wherein changing one or more of the stimulationfrequencies based on the feedback happens in real-time or near real-timeduring the user's current regimen.
 9. The system of claim 6, wherein thechanging of one or more of the stimulation frequencies based on feedbackis applied to a future regimen of the user.
 10. The system of claim 1,further comprising an external transducer selected from the list of anaudio speaker, an audio headphone, a haptic headband, a vibrating gamecontroller, an ultrasonic transducer, a radio frequency stimulator, amagnetic stimulator, transcranial direct current stimulation electrodes,and a deep brain stimulator, wherein at least one of the one or morestimulation frequencies is an operating frequency for the externaltransducer.
 11. A method for therapeutic stimulation using virtualobjects, comprising the steps of: receiving, at a stimulus manager, atherapeutic stimulation regimen; selecting one or more stimulationfrequencies based on the regimen; receiving data from a virtual reality(VR) application, the data comprising a location of a virtual objectdisplayed on a display; based on the regimen, instructing the VRapplication to change a visual state of the virtual object on thedisplay at the selected stimulation frequencies; and changing one ormore of the stimulation frequencies based on feedback, wherein thefeedback comprises determination of a user's attention based on theuser's interaction with the virtual object.
 12. The method of claim 11,further comprising the steps of: operating a virtual environment on acomputing device, the virtual environment comprising the virtual objectdisplayed on the display; receiving the instruction to change the visualstate of the virtual object; and changing the visual state of thevirtual object on the display at the selected stimulation frequencies.13. The method of claim 11, wherein the user's interaction with thevirtual object comprises user eye movement data captured using one ormore eye tracking sensors.
 14. The method of claim 13, wherein theregimen comprises one or more schemes for evaluating one or moreattention sub-processes.
 15. The method of claim 14, wherein theattention sub-processes comprise at least attention bias, selectiveattention, divided attention, sustained attention, and alternatingattention.
 16. The method of claim 15, wherein the stimulus manager isfurther configured to use the user eye movement data to determine anattention sub-score for each applicable attention sub-process.
 17. Themethod of claim 16, wherein the determination of the user's attention isbased on an aggregation of each of the attention sub-scores.
 18. Themethod of claim 16, wherein changing one or more of the stimulationfrequencies based on the feedback happens in real-time or near real-timeduring the user's current regimen.
 19. The method of claim 16, whereinthe changing of one or more of the stimulation frequencies based onfeedback is applied to a future regimen of the user.
 20. The method ofclaim 11, further comprising providing stimulation to the user with anexternal transducer selected from the list of an audio speaker, an audioheadphone, a haptic headband, a vibrating game controller, an ultrasonictransducer, a radio frequency stimulator, a magnetic stimulator,transcranial direct current stimulation electrodes, and a deep brainstimulator, wherein at least one of the selected plurality of stimuli isan operating frequency for the external transducer.